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Abstract 

We present results of a forecast initiated following assimilation of observations for week 
Week 50 (i.e. the forecast begins December 16, 2012) of the 2012-2013 influenza season for 
municipalities in the United States. The forecast was made on December 21, 2012. Results 
from forecasts initiated the three previous weeks (Weeks 47-49) are also presented. Also results 
from forecasts generated with the SIRS model without absolute humidity forcing (no AH) are 
shown. 



1 Background 

Forecast skill is calculated for individual cities, as well as for census divisions (regions) and all 
cities in aggregate, from retrospective forecasts made for the 2003-2004 through 20 11 -20 12 seasons, 
excluding the pandemic years 2008-2009 and 2009-2010 ( which will need to be handled separately 
in the future). The forecast methods are similar to those described in Shaman and Karspeck] (20 12 1. 



Based on the relationship between prediction accuracy and ensemble spread of these retrospective 
forecasts we can assign calibrated confidences to our current predictions. 

Figure [T] shows the results for all cities in aggregate using climatological AH and a factor of 5 
mapping. Overall the relationship is informative; however, for all lead times there is a basic plateau 
of skill once the ensemble log variance drops below 2.5 to 3 weeks^. 

When the cities are grouped by region, there is some heterogeneity. Some regions (e.g. West 
North Central) show marked improvement of forecast accuracy/skill with decreasing spread across 
all lead times. Other regions show much more limited skill-the Mountain region only has skill at 
1-3 weeks, and the East North Central has problems at 1-3 weeks. 



* Corresponding author address: Jeffrey Shaman, Department of Environmental Health Sciences, Mailman School of 
Public Health, Columbia University, 722 West 168th Street, Rosenfield Building, Room 1104C, New York, NY 10032. 
E-mail: jlsl06@columbia.edu 
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Figure 1 : Plot of ensemble mode forecast accuracy versus ensemble spread measured as log ensem- 
ble variance +1. Left) 114 cities in aggregate. The runs are binned in increments of 0.25 units and 
stratified by forecast lead time: 1-3 weeks (red), 4-6 weeks (cyan), 7-9 weeks (blue), lO-i- weeks 
(magenta). Right) Same as left, but the 1 14 cities grouped by census region. 



2 2012-2013 Forecast 

This record is part of an evolving series of real-time forecasts developed during the 2012-2013 
influenza season for the United States. Additional documentation of earlier forecasts for this season 



have also been posted ( Shaman et al. 2012) 



2.1 Week 50 Forecast 

The Week 50 forecast (initiated December 16, 2012) predicts peaks for Atlanta Week 49, 1 week in 
the past (Figure|2]). This is consistent with the prior week forecast, so the forecast for Atlanta remains 
unchanged. The forecast for Chicago is for either 1 week in the past or the week just finished (Week 
49 or 50); the histograms for Chicago reveal that week 50 is favored (Figure |3]), which is a slight 
shift from the prior week prediction, which was firmly on Week 49 (see Figures |5] and |6]). 

Dallas has shifted slightly from a Week 49 forecast of 0- 1 weeks in the future (either Week 49 or 
50) to a Week 50 forecast of weeks in the future (Week 50). Last week Houston was predicted to 
peak Week 50, and this prediction remains unchanged. Memphis is now predicted to peak in Week 
51 (last week it was Week 50), though this really reflects a shift in the distribution of predictions 
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from majority Week 50 and some Week 51 (Figure |6]) to majority Week 51 and some Week 50 
(Figure [3]). 

The Atlantia, Chicago, Dallas, Houston and Memphis predictions all come with high confidence 
(50-99%, depending on calibration, Figure[2]). The prediction for St. Louis is for Week 51; this is a 
shift from Week 50 last week, though as for Memphis, the histograms reveal a less marked shift. 

Miami is still predicted to peak in Week 51. New York City is predicted to peak during Week 
52, whereas last week it was Week 51. As for Memphis and St. Louis, the shift is not a step but 
a shift in the distribution of oucomes. Both Los Angeles and San Francisco are predicted to peak 
during Week 52, which is consistent with the prior week prediction of Weeks 51-52. Denver, too, is 
predicted to peak Week 52, as before. Seattle is predicted to peak 2-3 weeks in the future (Weeks 
52-53); this is consistent with the prior week prediction. 

All changes in predicted peak week timing are within the margin of error of ±1 week. Table 
[T] presents the predictions made over each of the last 4 weeks. It can be seen that the real-time 
predictions have been fairly stable over the last 4 weeks. In particular, Miami, Atlanta, Chicago, and 
St. Louis, haven't moved at all (essentially). Los Angeles, Denver, Dallas, and Houston have drifted 
1 week. San Francisco, Memphis and Seattle have moved 1.5 weeks. New York City has move a 
bit more, and unlike the other cities, the predicted peak has moved closer and then farther into the 
future. Most other predictions, which have drifted, have drifted farther into the future. 

Table 1: Summary of weekly model predictions at 12 select cities. Weeks are labeled consecutively 
(Week 1 of 2013 is Week 53, etc.). Predictions were initiated at the end of Weeks 47, 48, 49 and 
50. Figures present histogram distributions of the predictions. The range of prediction confidences, 
derived from municipal, regional and national calibrations, are given in parentheses. 



City 


Week 50 


Week 49 


Week 48 


Week 47 




Prediction 


Prediction 


Prediction 


Prediction 


Los Angeles 


52 (50-95%) 


51-52 (35-90%) 


51-52(20-55%) 


51 (15-30%) 


San Francisco 


52 (35-85%) 


51-52 (25-40%) 


51 (30-85%) 


50-51 (25-60%) 


Denver 


52 (20-60%) 


52 (20-55%) 


51-52 (0-55%) 


51 (10-30%) 


Miami 


51 (40-60%) 


51 (40-99%) 


50-51 (40-55%) 


50-51 (0-45%) 


Atlanta 


49 (80-99%) 


49 (90-99%) 


49 (80-95%) 


49 (80-95%) 


Chicago 


49-50 (55-95%) 


49 (55-95%) 


49 (35-80%) 


49 (35-80%) 


St. Louis 


51 (85-99%) 


50-51 (80-99%) 


50 (85-99%) 


51 (30-90%) 


New York City 


52 (25-99%) 


51 (25-99%) 


52-53 (25-60%) 


53-54 (25-55%) 


Memphis 


51 (20-80%) 


50 (20-80%) 


50(15-80%) 


49-50 (15-55%) 


Dallas 


50 (65-90%) 


49-50 (65-85%) 


49 (50-75%) 


49 (40-85%) 


Houston 


50 (75-90%) 


50 (50-60%) 


50 (50-60%) 


49 (50-85%) 


Seattle 


52-53 (0-55%) 


52-53 (5-55%) 


51-52 (5-55%) 


51 (5-35%) 



Prediction confidences have shifted only a bit. For instance, Seattle has low confidence in all 
predictions; Atlanta and St. Louis have been high. Dallas, has increased through time. Figure |4] 
shows time series of the ILI-i- metric for the 12 focus cities. This is the evolving target, though actual 
observed peak week will not be known until the outbreaks have abated. 
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2.2 Week 49 Forecast 



The Week 49 forecast (initiated December 9, 2012) basically stays on track with predictions made 
in prior weeks. Atlanta and Chicago are all predicted to be at peak (±1 week) during week 49 
(Figure|5]), which is the week ending December 8, 2012. Basically, these forecasts predict no future 
week higher then that latest observed week. Dallas is forecast to predict in 0-1 weeks. Houston and 
Memphis are forecast to peak in one week (the week ending December 15, 2012). The calibrated 
confidence in these predictions is fairly high (> 50%, except Memphis at the municipal scale). 

St. Louis is predicted to peak in 1-2 weeks. Miami is now predicted to peak in 2 weeks, as is 
New York City. The New York City prediction is a change of 2-3 weeks from the prior week (Week 
48) prediction of peak in 4-5 weeks. Both Los Angeles and San Francisco are forecast to peak in 2-3 
weeks. Denver is predicted to peak in 3 weeks (with about 50% confidence). Seattle is predicted to 
peak in 3-4 weeks. Figure [6] shows histograms of these predictions. 

Most of these changes are shifts in the prediction of 1 week from the prior week prediction, 
indicative of a similar tracking of outbreak evolutions. New York City had a larger shift of 2-3 
weeks. (Indeed, two weeks ago. New York City had a peak predicted to be 6-7 weeks in the future; 
so the forecast trajectory has shifted.) From the histograms, it can be seen that overall for New York 
City, there is less spread among the forecasts within an ensemble-the histogram is tightly spread 
between 2-3 week leads, whereas for the week prior (not shown) there was greater spread in the 
predicted peak weeks. 

2.3 Week 50 Forecast - No AH 

Forecasts initiated after assimilation of Week 50 observations at the beginning of Week 51 (Decem- 
ber 16, 2012) using an SIRS model without absolute humidity forcing produce generally similar 
predictions (Figure |7]). These similarities include predicted outbreak peaks of Week 49 for Atlanta 
and Chicago, Week 50 for Memphis and Houston, Week 51 for St. Louis, and Week 52 for Denver. 
All other prediction differ by at most 1 week from those of the model with AH forcing. 

The predictions for the 12 focus cities using the SIRS model without AH forcing are summarized 
in Table [2]). 
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Table 2: Summary of weekly model predictions at 12 select cities using an SIRS model without 
absolute humidity forcing. Weeks are labeled consecutively (Week 1 of 2013 is Week 53, etc.). Pre- 
dictions were initiated at the end of Weeks 48, 49 and 50. Figures present histogram distributions of 
the predictions. The range of prediction confidences, derived from municipal, regional and national 
calibrations, are given in parentheses. 



City 


Week 50 


Week 49 


Week 48 




Prediction 


Prediction 


Prediction 


Los Angeles 


51 (25-50%) 


50-51 (25-50%) 


50 (25-50%) 


San Francisco 


51 (30-60%) 


50-51 (30-60%) 


50 (30-50%) 


Denver 


52 (50-80%) 


51-52 (40-85%) 


51 (40-60%) 


Miami 


50-51 (40-80%) 


50 (10-99%) 


50 (5-65%) 


Atlanta 


49 (80-99%) 


49 (90-99%) 


49 (25-95%) 


Chicago 


49 (55-95%) 


49 (55-95%) 


49 (25-65%) 


St. Louis 


51 (80-95%) 


50 (80-95%) 


50 (35-95%) 


New York City 


51-52 (30-60%) 


51 (30-60%) 


52-53 (25-60%) 


Memphis 


50 (45-99%) 


50 (10-90%) 


49-50 (15-55%) 


Dallas 


50 (65-90%) 


49 (15-85%) 


49 (40-80%) 


Houston 


50 (80-95%) 


50 (60-70%) 


49-50 (30-70%) 


Seattle 


51-52 (20-50%) 


51-52 (20-45%) 


51 (0-50%) 
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Figure 2: Ensemble mode peak week predictions initiated December 16, 2012, following assimila- 
tion of Week 50 observations, for 12 cities plotted as a function of probability/confidence calibrated 
from historical city, regional and national prediction accuracy. 
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Figure 3: Left) Histograms of the best ensemble start date trainings for forecasts made beginning the 
start of Week 51 (December 16, 2012) for select cities. The distributions show the ensemble spread 
among peak predictions. 




Figure 4: Time series of Weeks 38-50 ILI+ for the 2012-2013 season. ILI-i- is Google Flu Trends 
weekly municipal ILI estimates times CDC census division seropositive rates.. 



7 



■D 

u 
'•5 



a. S 

E 

in 
c 
m 



Probabilities Calibrated 
to Local Past Predictions 



52 > o>ot oo t o 



2 0,4 0.6 0.6 

Probabiiity is Accurate 



Probabilities Calibrated 
to Regional Past Predictions 



It N M 



0-2 0.4 6 0,8 

Probability is Accurate 



Probabilities Calibrated 
to National Past Predictions 



S 
u 

|£ 
E 

10 

c 



m m 



• o 



0,2 0.4 O.S 0.8 

Probability is Accurate 



o 


Los Angeles 


o 


Sar Francisco 


o 


Denver 


o 


Miami 


o 


Atiania 


o 


Chicago 


It 


St. Louis 


It 


New York City 


H 


Memphis 


X 


Dallas 




Houstor 


H 


Seattle 



Figure 5: Ensemble mode pealc weelc predictions initiated December 9, 2012, following assimilation 
of Week 49 observations, for 12 cities plotted as a function of probability /confidence calibrated from 
historical city, regional and national prediction accuracy. 
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Figure 6: Left) Histograms of the best ensemble start date trainings for forecasts made beginning 
the start of Week 50 (December 9, 2012), following assimilation of Week 49 observations, for select 
cities. The distributions show the ensemble spread among peak predictions. 
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Figure 7: Ensemble mode peak week predictions initiated December 16, 2012, following assimila- 
tion of Week 50 observations using an SIRS model without AH forcing, for 12 cities plotted as a 
function of probability/confidence calibrated from historical city, regional and national prediction 
accuracy. 
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